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I- Real Party iB Interest (37 CF.R§41.37(c)(l)(i)) 

The real paity in interest in the present appeal is Microsoft Coiporation, the assignee of 
the present application. 

Related Appeals and Interferences (37 C.F.IL §41.37(c)(l)(ii)) 
Appellants, appellants* legal representative, and/or the assignee of the present application 
are not aware of any appeals or interferences which may be related to, will directly affect, or be 
directly affected by or have a bearing on the Board's decision in the pending appeal. 

in. status of Claims (37 CF,R. §41.37(c)(l)(m)) 

Claims 11-14, 21-29, 41, 42 and 48-51 have been withdrawn. Claims 2 and 3 have been 
canceled. Claims 1, 4-10, 15-20, 30-40, 43-47, 52 and 53 stand rejected by the Examiner. The 
rejection of claims 1, 4-10, 15-20, 30-40, 43-47, 52 and 53 is being 25)pealed. 

IV. Status of Amendments (37 C.F.R. §41.37(c)(l)(iv)) 

The Examiner has entered amendments submitted after the Final Office Action. (See 
Coxnmunication from Examiner, Advisory Action dated July 25, 2005, page 1). 

V, Summary of Claimed Subject Matter (37 C.KR. §41.37(c)(l)(v)) 
A. Independent Claim 1 

Independent claim 1 recites, "A system that facilitates non-linear viewing of media, the 
system comprising: a scene selector that scans a digitized media and selects a scene in the 
digitized media based on at least one of, face recognition, item recognition, voice recognition, 
color recognition, mood recognition, theme recognition, and an input from a user; a metadata 
generator that produces metadata associated with the selected scene and annotates the selected 
scene with the metadata; and an organizer that places the selected annotated scene in a media 
store to facilitate non-linear viewing of one or more scenes." {See e.g., FIG. 1; page 4, lines 16- 
24; page 7, lines 9-11). 
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B. * Independent Oaim 6 

Independent claim 6 recites, "A method that facilitates non-linear viewing of media, 
comprising: selecting a scene from a set of digitized media based on at least one of, face 
recognition, item recognition, voice recognition, color recognition, mood recognition, theme 
recognition, and an input from a user; annotating the selected scene with metadata; and storing 
the annotated scene to facilitate non-linear retrieval of the annotated scene.*' {See eg., FIG. 2; 
page 6, line 26 - page 7 line 25), 

C. Independent Claim 15 

Independent claim 15 recites, "A system for non-linear viewing of media, comprising: a 
scene retriever that retrieves one or more annotated scenes and one or more pieces of annotating 
metadata associated with the one or more annotated scenes from a media store; a metadata 
analyzer that analyzes the one or more pieces of annotating metadata to identil^ one or more 
relationships between the one or more pieces of annotating metadata; a playlist generator that 
evaluates the one or more relationships and produces a playlist of related scenes." {See e.g. , FIG. 
3; page 8, line 3 - page 9, line 27). 

D. Independent Claiin 30 

Independent claim 30 recites, "A system for annotating video media, comprising: a media 
database comprising a playlist and one or more video segments, where the video segments are 
associated with an aimotating metadata based on at least one of, face recognition, item 
recognition, voice recognition, color recognition, mood recognition, theme recognition, and an 
input from a user; and an annotating tool that facilitates creating the annotating metadata based 
on at least one of, face recognition, item recognition, voice recognition, color recognition, mood 
recognition, theme recognition, and an input from a user and annotated video segments by 
associating the annotating metadata with the video segments." {See e.g., FIG. 8; page 7, lines 9- 
11; page 15, line 11 - page 16, line 14). 

E. Independent Claim 35 

Independent claim 35 recites, "A system for delivering media content, the system 
comprising: a media data store comprising one or more metadata-annotated displayable items; a 
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presenter that presents a selected first displayable item from the media data store; and a selector 
that selects a second displayable item from the media data store based, at least in part, on a 
relationship between a first metadata associated with the first displayed item and a second 
metadata associated with the second displayable item." (See e.g., FIG. 9; page 16, line IS - page 
17, line 7). 

F. Independent Claim 43 

Independent claim 43 recites, "A data structure that facilitates non-linear viewing of 
media items, the data structure comprising: a first field that holds a media item; and a second 
field that holds a metadata item related to the media item, where fhe metadata facilitates locating 
a related media item by annotating the related media with metadata, and where the metadata 
facilitates at least one of identifying the media item.'* (See e.g., FIG. 12; page 19, lines 3-19; 
page 19, lines 13-19). 

G. Independent Claim 44 

Independent claim 44 recites, "In a computer system having a graphical user interface 
that comprises a display and a selection device^ a meUiod of providing and selecting from a set of 
graphical user interface elements on the display, the method comprising: retrieving a set of 
graphical user interface elements, each of the interface elements representing an action 
associated with fricilitating the non-linear display of media items; displaying the set of interface 
elements on the display; receiving an interface element selection signal indicative of the 
selection device selecting a selected entry from the set of interface elements; and in response to 
the inter&ce element selection signal, initiating processing of related media by the metadata 
generator and to facilitate non-linear viewing of media based, at least in part, upon stored 
metadata." {See FIG. 13; page 19, line 20 - page 20, hne 21). 

H. Independent Claim 45 

Independent claim 45 recites, "A computer-readable medium having computer executable 
instructions for employing a data packet adapted to be transmitted between two or more 
computer components that facilitate annotating a media and facilitate the non-linear viewing of 
the media, the data packet comprising: a first field that stores a clip identifier that identifies a 
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portion of a media; a second field that stores a metadata key that identifies an annotating 
metadata associated with the clip identified by the clip identifier; and a third field that stores data 
associated with the clip identified by the clip identifier.** {See e,g,, FIG. 14; page 20, line 22 - 
page 22, line 2). 

I, Independent Claim 46 

Independent claim 46 recites, ''A computer-readable medium having computer executable 
instructions for employing a memory for storing data for access by a computer component, the 
memory comprising: a data structure stored in the memory, the data structure holding: a first 
field that stores a clip identifier that identifies a portion of a media; a second field that stores a 
metadata key that identifies an annotating metadata associated with the clip identified by the clip 
identifier; and a third field tiiat stores data associated with the clip identified by the clip 
identifier." (See e.g., FIG, 14; page 20, line 22 - page 22, line 2). 

J. Independent Claim 47 

Independent claim 47 recites, '^A computer*readable medium having computer executable 
instructions for employing a data packet adapted to be transmitted between two or more 
computer components that facilitate the non-linear viewing of a media, the data packet 
comprising: a first field that stores a clip identifier that identifies a portion of a medi^ a second 
field that stores a requested user action concerning the portion identified by the clip id^tifier; 
and a third field that stores metadata associated with the portion identified by the clip identifi^, 
where the metadata is employed to adapt one or more clips according to fiie requested user 
action," (See e.g,, FIG. 14; page 20, line 22 - page 22, line 2). 

VI. Grounds of Rejection to be Reviewed (37 C.F.R. §41.37(c)(l)(vi)) 

A. Whether claims 45-47 are unpatentable under 35 U.S.C. § 101 . 

B* Whether claims 1-6, 8-10, 43, 44, 52 and 53 are unpatentable under 35 U.S.C. 
§102(e) over Jain, et al. (U.S. 6,567,980). 
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C Whether claims 45-47 are unpatentable under 35 U.S.C, §102(e) over Morris 
(U.S. Patent Application: 2002/0088000). 



D. Whether claim 7 is unpatentable under 35 U.S.C. §103(a) over Jain, et al (U.S. 
6,567,980) in view of Buncombe (U.S. 6,813,745). 



Whether claims 15-20 and 30-40 are unpatentable under 35 U.S.C- §103(a) over 
Jain, etal in view of Duncombe (U.S. 6,792,573). 



m Argument (37 C.F.R. §4137(c)(l)(viO) 



A- Rejection of Claims 45-47 Under 35 U,S.C SlOl 

Claims 45-47 stand rejected under 35 U.S-C- §101. It is respectfiiUy submitted that this 
rejection is improper and should be reversed for at least the following reasons. The subject 
claims produce a useful, concrete, and tangible result and fall within a statutory class for 
inventions. 

Patentable subject matter is not limited to tangible articles or 
objects, but includes intangible subject matter, such as data or 
signals^ representative of or constituting physical activity or 
objects. In re Warmerdam, 33 R3d 1354, 1360, 31 U.S.P.Q.2D 
(BNA) 1754, 1760 (Fed. Cir. 1994) (citing //i re Schrader, 22 F.3d 
290, 295 (Fed. Cir. 1994) (emphasis added). Title 35, section 101, 
explains that an invention includes "any new and usefiil process, 
machine, manufacture or composition of matter."... Without 
question, software code alone qualifies as an invention eligible 
for patenting under these categories^ at least as processes. Eolas 
Techs,, Inc, v. Microsoft Corp., 399 F.3d 1325, 1338-39 (Fed. Cir. 
2005) (emphasis added). 



Independent claims 45-47 recite, ''A computer-readable medium having computer 
executable instructions". The Examiner incorrectly argues at page 2 of the Advisory Action that 
one possible embodiment of the computer-readable medium is a "carrier wave" (e.g., a signal), 
and therefore the claims are non-statutory. Appellants' representative respectfully submits that 
signal claims are within the provinces of statutory subject matter. See In re Warmerdam, supra, 
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Moreover, the recited computer executable instructions are software code . Pursuant to Eolas 
Techs,, Inc. v. Microsoft Corp,, software code aJpne qualifies as an invention eligible for 
patenting, even if intangible {e,g,, manifested without physical or structural components). 
Accordingly, this rejection of claims 45-47 should be reversed. 

B. Rejection of Claims 1-6, 8-1 0> 43. 44. 52 and S3 Under 35 U.S.C, 8102fe> 
Claims 1-6, 8-10, 43, 44, 52 and 53 stand rejected under 35 U,S.C. §102(e) a$ being 
anticipated by Jain, et aL (U.S. 6,567,980). Reversal of this rejection is respectfully requested 
for at least the following reasons. Jain, et al. fails to teach or suggest each and every limitation 
as recited in the subject claims. 

A single prior art reference anticipates a patent claim only if it 
expressly or inherently describes each and every limitation set 
forth in the patent claim. Trintec Industries, Inc., v. Top-U.S.A. 
Corp., 295 F.3d 1292, 63 U.S.P.Q.2D 1597 (Fed, Cir. 2002); See 
Verdegaal Bros. v. Union Oil Co. of California, 814 F.2d 628, 631, 
2 USPQ 2d 1051, 1053 (Fed. Cir. 1987) (emphasis added). The 
identical invention must be shown in as complete detail as is 
contained in the claim, Richardson v. Suzuki Motor Co.y 868 
R2d 1226, 9 USPQ2d 1913, 1920 (Fed. Cir. 1989) (emphasis 
added). 

Independent Claims I and 6 

The claimed invention relates to viewing annotated digital media and non-linear viewing 
of related annotated media. In particular, independent claim 1 (and similarly independent claim 
6) recites, "a metadata generator that produces metadata associated with the selected scene and 
annotates the selected scene with the metadata; and an organizer that places the selected 
annotated scene in a media store to faciUtate non-linear viewing of one or more scenes". Jain, et 
al. does not disclose or suggest these novel aspects of ^pellants' claimed invention. 

Rather, Jain, et al. discloses a multimedia cataloger that extracts metadata from video 
(see coL 2, lines 1 8-20), and commits the metadata to the Metadata Track Index Manager and/or 
a track index (see coL 8, lines 54-59; col. 9, lines 11-12). Jain, et al. neithw annotates digital 
media with metada.ta, nor stores annotated media as recited in the subject claims. Instead, die 
reference extracts metadata iBrom the media, and stores that metadata separately firom the media. 
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{See e.g., Fig. 1, items 130, 140). Consequently, the non-linear access to various segments of the 
media requires an intermediate index and is therefore distinct from the subject invention. 

At page 2 of the Advisory Action, the Examiner incorrectly asserts that extracting 
metadata from media is equivalent to annotating the media with metadata. It is readily ^parent 
that media directly annotated with metadata (i.e., annotated media) is distinct fronj unannotated 
media that has an associated index of associated metadata. For example, it is cumbersome to 
store media and metadata separately and also inejEficient to consult an intermediary to provide 
non-linear access, when, as provided for in the subject invention, annotated media combines the 
media with the metadata for manageable storage and for rapid non-linear access. Jain, et aL does 
not teach or suggest "a metadata generator that produces metadata associated with the selected 
scene and annotates the sele^ed scene with the metadttta; and an or ganis er that places the 
selected annotated scene in a media store to fecilitate'non-linear viewing of one or more 
scenes". Accordingly, Jain, et aL feils to teach or suggest the identical invention in as complete 
detail as is contained in the subject claims, and this rejection should be reversed. 

Independent Claims 43 and 44 

Indeprndent claim 43 recites, "a second field that holds a metadata item related to the 
media it^, where the metadata facilitates locating a related media item by annotttting the 
related media adth metadata^\ Independent claim 44 recites, "in response to the interface 
element selection signal, initiating processing of related media by the metattata generator to 
feciUtate non-linear viewing of media based, at least in part, upon stored metadata", wherein the 
naetadata generator as described by flie disclosure annotates media with metadata. (See page 4, 
lines 16-18). As detailed, supra, Jain, et aL does not provide for annotating the related media 
with metadata, and merely extractijig metadata that is associated with the media is insufficient to 
read upon the claimed invention. Accordingly, reversal of this rejection with respect to 
independent claims 1, 6, 43, and 44, as well as all claims that depend there from, is respectfully 
requested. 
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C. Rejection of Claims 45'-47 Under 35 U.S>C, S102fe^ 

Claims 45-47 stand rejected under 35 U.S.C. § 102(e) as beiiig anticipated by Morris 
(U.S. Patent Application: 2002/0088000). Reversal of this rejection is respectfully requested for 
at least the following reasons. Moms fails to teach or suggest each and every limitation set forth 
in the subject claims. 

Independent claims 45 and 46 recite, "a second field that stores a metadata key that 
identifies an annotating metadata associated with the clip identified by the clip identifier/' 
Independent claim 47 recites, "a second field that stores a requested user action . . , and a third 
field that stores metadata. . .where the metadata is employed to adapt one or mote cUps 
according to the requested user action^" Morris does not disclose or suggest these limitations. 

Rather, Moiiis relates to a method and system for controlling access to image metadata. 
Digital images captured and stored on an internal memory of a digital camera can be uploaded to 
a database for storing the images [and the metadata]. {See Paragraphs 0017-0018), This 
metadata is stored within the image file in individual image tags. {See Fig. 2, item 60; paragraph 
0020). The tags include system tags that describe such tliingfi as focus setting, ^erture setting, 
etc, (see Fig. 3, item 64; paragraph 0021). and user tags that are used to store labels such as 
'Tjirthday or 'Vacation" (See Fig. 3, item 66; paragraph 0022). Morris is directed toward 
providing access to metadata (see e.g., paragraph 0005, ^'Friends and family may then access the 
photo website and not only see the pictures, but read and/or hear the story and history behind 
each image by accessing the metadata!")^ but is silent on whether the user actions (as opposed to 
descriptive labels) are stored within the metadata, and also silent on whether metadata can be 
employed to adapt a clip. Accordingly, contrary to the Examiner's contention, user tags neither 
store a requested user action, nor are the metadata employed to adapt one or more clips 
according to the requested user action^ as recited in the subject claims. Raflier, the reference 
stores only labels, and these labels are merely descriptive, without any fimctional ability 
disclosed, let alone the ability to adapt a clip according to a user action. 

Furthennore, tiie data stmcture of Morris stores actual metadata, not a metadata key^ as 
recited in independent claims 45 and 46. Morris does not teach or suggest a metadata key. 
Rather, Morris discloses an unsophisticated method of storing individual image files and is silent 
on whether metadata for one image can relate to other metadata (e.g., metadata in the same or a 
different image). Therefore, contrary to the Examiner's remarks at page 13 of the Final Office 
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Action (dated June 1, 2005); "identification of metadata" is insufficient to read upon a metadata 
key. For example, a key has dynamic, relational aspects with other data elements. The 
simplistic disclosure of the reference {e.g. , that metadata can be stored in fields) does not 
contemplate the use of metadata keys, and, hence, does not anticipate the subject claims. 
Accordingly, this rejection of independent claims 45-47 should be reversed. 



D. Reiectioii of Claim 7 Under 35 U,S.C. S103(aj 

Claim 7 stands rejected under 35 U.S.C. § 103(a) as being unpatentable over Jain» et al. 
(U.S. 6,567,980) in view of Duncombe (Q,S. 6,813,745). Reversal of this rejection is 
respectfully requested for at least the followmg reasons. Claim 7 depends directly from 
independent claim 6, and Duncombe, which relates to a media system for storing media files and 
a media organization file, does not make up for the aforementioned deficiencies of Jain, et ah 
with respect to claim 6, Accordingly, this rejection should be reversed. 

E. Rejection of Claims 15-20 and 30-40 Under 35 U.S-C. S103(a^ 

Claims 15-20 and 30-40 stand rejected under 35 U.S.C. §103(a) as being unpatentable 
over Jain, et al in view of Duncombe (U.S. 6,792,573). Reversal of this rejection is respectfully 
requested for al least the following reasons. Jain, et aL, alone and/or in combination with 
Duncombe, does not teach or suggest all limitations set forth in the subject claims. 

To reject claims in an application under §103, an examiner must 
establish a prima facie case of obviousness, A prima facie case of 
obviousness is estabhshed by a showing of Ihree basic criteria. 
First, there must be some suggestion or motivation, either in the 
references themselves or in the knowledge generally available to 
one of ordinary skill in the art, to modify the reference or to 
combine reference teachings. Second there must be a reasonable 
expectation of success. FinaUy, the prior art reference (or 
references when combined) must teach or suggest all the claim 
limitations. The teaching or suggestion to make the claimed 
combination and the reasonable expectation of success must be 
found in the prior art and not based on the Applicants* disclosure. 
See Jn re VaecK 947 F.2d 488, 20 USPQ2d 1438 (Fed. Cir. 1991). 
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Appellants' claimed invention further relates to annotating digital media with metadata to 
facilitate non-linear viewing of related scenes that are annotated with metadata. In particular, 
independent claim 15 (and similarly independent claims 30 and 35) recites, "a scene retriever 
that retrieves one or more annotated scenes'* and "a playlist generator that evaluates the one or 
more relationships and produces a playlist of related scenes". The Examiner concedes that Jain, 
et al. does not teach or suggest every limitation, but contends that Duncombe will cure the 
deficiencies. 

However, as detailed above, independent claims 15, 30 and 35 recite limitations similar 
to independent claim 1, which are not disclosed or suggested by Jain, et al. In particular, the 
instant claims recite annrifaterf media scenes, video segments, and displayable items). Jain, 
et aL does not teach annotated media, and Duncombe does not make up for tiiis deficiency. 
Accordingly, the aforementioned claims as well as all associated dependent claims are therefore 
believed to be allowable over the cited references. 

Duncombe relates to a method for playing media based upon feedback from a user, and 
discloses the goal seeking engine first selects, fsom the media clips, only those that are 
associated with the topic that matches the desired topic; and, secondly, selects only those media 
clips with matching complexity ratings, order ratings, and/or time length. {See col. 8, lines 22- 
60). Hence, the goal sedcing engine requires at least two user inputs in order to locate relevant 
data {see col. 4, lines 53-54) and create or modify a list. Duncombe "organizes data by treating it 
like a four-dimensional object" {see col. 4, lines 19-20), but metadata is not contemplated by the 
reference. Therefore, relationships between metadata are not utilized to index the clips 
automatically, rather, the clips are organized by a "professional" {see col. 2, lines 60-64), and can 
be later reorganized based upon user ratings {see col. 2, lines 54-56) of predetermined categories. 
In essence, Duncombe selects media clips based upon relationships between the media as defined 
by a professional and/or users who have previously viewed die clip. Duncombe does not choose 
clips based upon relationships between one or more pieces of aimotating metadata . 

At pages 8-9, the Final Office Action concedes that Jain, et aL does not teach or suggest 
"a playlist generator that evaluates the one or more relationships and produces a playlist of 
related scenes" (wherein the relationships are between the one or more pieces of aimotating 
metadata), but contends that Duncombe will alleviate this insufficiency when combined with 
Jain, et aL However, "a playHst generator" from Duncombe the Examiner seeks to incorporate 
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into Jain, et al does not evaluate relationships between metadata, but instead evaluates based 
upon categorical ratings provided by a professional Therefore, the proposed combination does 
not teach or suggest all the claim limitations. 

The Examiner further argues at page 14 of the Final Office Action that *'a playlist 
generator" from Duncombe would then use metadata provided by Jain, et al. instead of using a 
professionally-organized and user-updated method to function as the disclosure teaches. In 
essence, the Examiner is not combining Jain, et aL with a playlist generator from Duncombe, but 
instead with a hypothetical playlist generator that works in the manner the Examiner desires in 
order to reject the subject claims. For example, Jain, et al. does not disclose a playlist generator 
(as the Examiner concedes), thus, the reference cannot teach or suggest a playlist generator that 
evaluates the one or more relationships (between the annotating metadata). Duncombe teaches a 
playlist generator, but hot one that evaluates the one or more relationships (between metadata). 
Accordingly, neither reference teaches a playlist generator that evaluates the one or more 
relationships (between metadata), yet the Examinar, has taken the concept of a playlist generator, 
and given it attributes that neither reference discloses. In particular, the Examiner suggests a 
playlist generator from Duncombe would then use metadata provided by Jain, al, ostensibly 
because Jain, et al. discloses the use of metadata in connection with other features. 

Assuming arguendo that Jain et aL evaluates relationships between metadata, the 
reference does not teach or suggest doing so in connection with a playlist generator. It is not 
enough to simply postulate that tbie combination could be made, and would then work in the 
manner the Examiner desires simply because other aspects of the combined device provide for 
such fl^ibility. See In re Fritch, 972 R2d 1260, 1265-66 (Fed. Cir. 1992) (holding that 
flexibility shown in one aspect of the reference does not imply extending that flexibility to die 
entire device). The Examiner has failed to make a prima facie case for obviousness and this 
rejection of independent claims 15, 30, and 35, as well as all claims that depend there from, 
should be reversed. 
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F- Conclusion 

For at least the above reasons, the claims currently xmder consideration axe believed to be 
patentable over the cited references. Accordingly, it is respectfully requested that the rejections 
of claims 1, 4-10, 15-20, 30-40, 43-47, 52 and 53 be reversed. 

If any additional fees are due in connection with this document, the Commissioner is 
authorized to charge those fees to Deposit Account No. 50-1063 [MSFTP304US], 



AMIN & TUROCY, LLP 

24*** Floor, National City Center 
1900 East 9"^ Street 
Telephone: (216)696-8730 
Facsimile: (216)696-8731 
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Vra. Claims Appendix (37 CF.R §41.37(c)(l)(viii)) 

1. A system that facilitates non-linear viewing of media, the system comprising: 

a scene selector that scans a digitized media and selects a scene in the digitized media 
based on at least one of, face recognition, item recognition, voice recognition, color recognition, 
mood recognition, theme recognition, and an input from a user; 

a metadata generator that produces metadata associated with the selected scene and 
annotates the selected scene with the metadata; and 

an organizer that places the selected annotated scene in a media store to facilitate non- 
Unear viewing of one or more scenes. 

2. (Canceled) 

3. (Canceled) 

4. The system of claim 1 , where at least one of the face recognition, item recognition, voice 
recognition, color recognition, mood recognition and theme recognition is ad^ted by a machine 
learning technique based, at least in part, on the input jfrom the user. 

5. The system of claim 1, where tfie metadata generator produces at least one of, a date, a 
time, a length, a subject, a mood, a theme, a color, a person name, a set of p^^n names, an item 
name and a set of item names associated witti the scene. 

6. A method that facilitates non-linear viewing of media, comprising: 

selecting a scene from a set of digitized media based on at least one of, face recognition, 
item recognition, voice recognition, color recognition, mood recognition, theme recognition, and 
an input from a user; 

annotating the selected scene with metadata; and 

storing the amiotated scene to facilitate non-linear retrieval of the annotated scene. 
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7. The method of claim 6, where selecting the scene from the set of digitized media 
comprises: 

mamially scanning one or more scenes from the set of digitized media; and 
manually selecting the scene. 

8. The method of claim 6, where selecting the scene from the set of digitized media 
comprises: 

automatically scanning one or more scenes from the set of digitized media; and 
automatically selecting the scene based on at least one of face recognition, item 
recognition, voice recognition, color recognition, mood recognition and theme recognition. 

9 . The method of claim 6, where annotating the selected scene with metadata comprises 
associating at least one of, a date, a time, a length, a subject, a mood, a theme, a color, a person, a 
set of people, an item and a set of items with the selected scene. 

10. The method of claim 6, where storing the annotated scene to facilitate non-linear retrieval 
of the annotated scene comprises storing the annotated scene in at least one of, a database and a 
datacube. 

11 - A system for generating a playlist of media items to facilitate non-linear viewing of the 
media items, comprising: 

a scene retriever that retrieves one or more scenes and one or more pieces of annotating 
metadata associated with the one or more scenes from a media store; 

a metadata analyzer that analyzes the one or more pieces of annotating metadata to 
identify one or more relationships between the one or more pieces of annotating metadata; and 

a playlist generator that evaluates the one or more relationships and produces a playlist of 
related scenes. 

12. The system of claim 1 1 , where the scene retriever retrieves a scene based, at least in part, 
on a value stored in the one or more pieces of annotating metadata. 
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13. The system of claim 12, where the metadata analyzer computes one or more similarity 
values for the one or more relationships. 

14. The system of claim 13, where the playlist generator produces the playhst of related 
scenes based, at least in part, on the similarity values. 

15. A system for non-linear viewing of media, comprising: 

a scene retriever that retrieves one or more annotated scenes and one or more pieces of 
annotating metadata associated with the one or more annotated scenes from a media store; 

a metadata analyzer that analyzes tiie one or more pieces of annotating metadata to 
identify one or more relationships between the one or more pieces of annotating metadata; 

a playhst generator that evaluates the one or more relationships and produces a playlist of 
related scenes. 

one or more viewers for viewing a scene hsted in the playlist; 

one or more feedback receivers for receiving a feedback concerning the viewed scene; 

and 

a playlist updater for updating the playhst based, at least in part, on tiie feedback. 

16. The system of claim 15, where the one or more viewers comprise at least one of, an 
active device and a passive device. 

1 7. The system of claim 1 5, where ttie one or more viewers comprise at least one of, an 
inteUigent device and a non-intelUgent device. 

1 8. The system of claim 1 5, where the feedback comprises at least one of, a touch input, a 
typed input, a mouse input, a voice input and a facial expression input coDceming the viewed 
scene. 
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19. The system of claim 18, where the feedback concerns a current scene and where the 
feedback comprises at least one of, a command to skip ahead in the playlist, a command to skip 
back in the playlist, a command to generate a new playlist, a command to find scenes similar to 
the current scene and a coiranand to play a longer scene related to the current scene. 

20. Tlie system of claim 15, where the playlist updater adds and/or removes a scene fiom the 
playlist based on at least one of, a usage data, a feedback command and a time stamp. 

21 . A method for facilitating non-linear viewing of media, comprising: 
selecting one or more stored scenes to view; 

analyzing one or more first metadata associated with the one or more selected scenes; and 
generating a playlist of stored scenes by evaluating one or more relationships betweeo the 
one or more first metadata and one or more second metadata. 

22. The method of claim 21 wh^e selecting one or more stored scenes to view comprises at 
least one of, selecting a scene based on the presence of a face in the scene, selecting a scene 
based on the absence of a face in the scene, selecting a scene based on the presence of an item in 
the scene, selecting a scene based on ^e absence of an item in die scene, selecting a scene based 
on a voice in the scene, selecting a scene based on the absence of a voice in the scene, selecting a 
scene based on a mood of the scene and selecting a scene based on the theme of ilie scene. 

23. The method of claim 22 where analyzing the one or more first metadata comprises 
computing one or more similarity scores for one or more pieces of metadata that hold 
information concerning at least one of a present face, a present item, a present voice, a mood and 
a theme. 

24. The method of claim 23 where generating the playlist comprises placing one or more 
scene identifiers in the playlist. 
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25. A method for non-linear viewing of media, comprising: 

displaying a scene stored in a playlist storing at least one scene aad metadata associated 
with the scene; 

receiving a user feedback related to the displayed scene; 
reacting to the user feedback; and 

if the user feedback commands an update to the playlist, selectively updating the playlist 
based, at least in part, on the user feedback. 

26. The method of claim 25, where receiving the user feedback comprises receiving at least 
one of, a touch input, a typed ii^ut, a mouse input, a voice input and a fiacial escpression input. 

27. The method of claim 25, where reacting to the user feedback comprises at least one of, 
moving forward in the playlist, moving backward in the playlist, and displaying a media item 
related to the scene. 

28. The method of claim 25, where updating the playlist comprises at least one of, adding a 
scene to the playlist and removing a scene trom the playlist 

29. A method for facilitating non-linear viewing of media, comprising: 
receiving feedback related to a viewed media scene; 

if the feedback requests finding related scenes, finding related scenes by analyzing a 
metadata associated with the viewed scene; 

if the feedback requests generating a new playlist of media scenes, calUng a method for 
generating a new playUst; and 

if the feedback requests navigating within a playUst, selectively skipping forward and/or 
backward in the playUst according to the feedback. 
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30. A system for annotating video media, comprising: 

a media database comprising a playlist and one or more video segments, where the video 
segments are associated with an annotating metadata based on at least one of, face recognition, 
item recognition, voice recognition, color recognition, mood recognition, theme recognition, and 
an input from a user; and 

an annotating tool that facilitates creating the annotating metadata based on at least one 
of, face recognition, item recognition, voice recognition, color recognition, mood recognition, 
theme recognition, and an input jfrom a user and annotated video segments by associating the 
annotating metadata with the video segments. 

3 1 . The system of claim 30, where the annotating metadata comprises at least one of, a date 
identifier, a time identifier, a videographer identifier, a face identifier, an item identifier, a voice 
identifier, a mood identifier and a theme identifier. 

32. The system of claim 30, where the annotating tool generates the annotating metadata in 
response to a user input. 

33 . The system of claim 30, wh^e flic annotating tool automatically generates the annotating 
metadata based, at least in part, on a face recognition, an item recognition data, a voice 
recognition data, a mood data and a th^e data. 

34. The system of claim 33, where the annotating tool is adapted based on a machine learning 
technique based, at least in part, on a user input concerning flie annotating metadata generated by 
the annotating tool. 

35. A system for delivering media content, the system comprising: 

a media data store comprising one or more metadata-annotated displayable items; 

a presenter that presents a selected first displayable item fix)m the media data store; and 

a selector that selects a second displayable item from the media data store based, at least 

in part, on a relationship between a first metadata associated with the first displayed item and a 

second metadata associated with the second displayable item. 
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36, The system of claim 35, where the media data store is at least one of a database, a data 
cube, a list, an array, a tree and a file. 

37- The system of claim 35, where the presenter is at least one of an intelligent display and a 
non-intelligent display. 

38. The system of claim 35, where the presenter is at least one of an active display and a 
passive display. 

39. The system of claim 35, where the selector selects a second displayable item based, at 
least in part, in response to a user response to the first displayable item. 

40. The system of claim 39, where the user response is at least one of, a spoken word, a 
keystroke, a mouse click, and a facial expression. 

41 . A media annotation and retrieval system, comprising: 

an annotator that annotates a set of media items with a metadata to facihtate retrieving 
media items related by die metadata; 

a playlist generator that generates a playlist of media items related by a first metadata 
retrieved in response to a first query; 

a presenter for presenting media items associated with the media item playlist; and 

a playlist updater that updates the playlist based on a second metadata retrieved in 
response to a second query, where the second metadata is related to the first metadata. 
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42. A method for annotating and retrieving media items, comprising: 

annotating a set of media items with a metadata to facilitate metadata based retrieval of 
one or more of the set of media items; 

receiving a first query related to identifying a media item by a relationship between the 
nledia item and a metadata; 

retrieving a first metadata responsive to the first query; 

generating a playlist of related media items where the media items are related by one or 
more metadata items in the first metadata; 

displaying one or more media items Usted in the playlist; 

receiving a second query related to identifying a media item by a relationship between the 
media item and a metadata; 

retrieving a second metadata responsive to the second query; and 
updating the playlist based on the second metadata. 

43. A data structure that fecilitates non-linear viewing of media items, the data stmcttire 
comprising: 

a first field that holds a media item; and 

a second field that holds a metadata item related to the media item, where the metadata 
facilitates locating a related media item by annotating the related media with metadata, and 
where the metadata facilitates at least one of identifying the media item. 
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44. In a computer system having a graphical user interface that comprises a display and a 
selection device, a method of providing and selecting j6pm a set of graphical user interface 
elements on the display, the method comprising: 

retrieving a set of graphical user interface elements, each of the interface elements 
representing an action associated with facihtating the non-linear display of media items; 
displaying the set of interface elements on the display; 

receiving an interface element selection signal indicative of the selection device selecting 
a selected entry from the set of interface elements; and 

in response to the interface element selection signal, initiating processing of related 
media by the metadata generator and to facilitate non-linear viewing of media based, at least in 
part, upon stored metadata. 

45. A computer-readable medium having computer executable instructions for employing a 
data packet adapted to be transmitted between two or more computer components that facilitate 
annotating a media and facilitate the non-linear viewing of the media, the data packet 
comprisdng: 

a first field that stores a clip identifier that identifies a portion of a media; 
a second field that stores a metadata key that identifies an annotating metadata associated 
with the clip identified by the clip identifier; and 

a third field that stores data associated with the clip identified by the clip identifier. 

46. A computer-readable medium having computer executable instmctions for employing a 
memoiy for storing data for access by a computer component, the memory comprising: 

a data structure stored in the memory, the data structure holding: 

a first field that stores a clip identifi^ that identifies a portion of a media; 

a second field that stores a metadata key that identifies an annotating metadata 

associated with the clip identified by the clip identifier, and 

a third field that stores data associated with the clip identified by the clip 

identifier. 
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47. A computer-readable medium having computer executable instructions for employing a 
data packet adapted to be transmitted between two or more computer components that facilitate 
the non-linear viewing of a media, the data packet comprising: 

a first field that stores a clip identifier that identifies a portion of a media; 
a second field that stores a requested user action concerning the portion identified by the 
clip identifier; and 

a third field that stores metadata associated with the portion identified by the clip 
identifier, where the metadata is employed to adapt one or more clips according to the requested 
user action. 

48. A set of application program interfaces embodied on a computer-readable medium for 
execution on a computer component in conjunction with a system for annotating and viewing 
media, comprising: 

a first inter&ce that receives media information; 

a second interface that receives annotation information associated with the media; and 
a third interface that receives user interface information associated with the order in 
which media will be displayed. 

49. A system for user-directed viewing of video, conq)ri$ing: 

means for partitioning a first video into one or more second videos; 
means for displaying the one or more second videos; and 

means for annotating the one or more second videos to facilitate retrieving at least one of 
the first video and the one or more of the second videos in response to a user input. 
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50. A computer readable medium storing computer executable components of a system for 
media amiotation and retrieval, comprising: 

an amiotating component that annotates a set of media items with a metadata to facilitate 
retrieving media items related by the metadata; 

a playlist generating component that generates a playlist of media items related by a first 
metadata retrieved in response to a first query; 

a presenting component for presenting media items associated with the media item 
playlist; and 

a playlist updating component that updates the playlist based on a second metadata 
retrieved in response to a second query, where the second metadata is related to the first 
metadata. 

51. A computer readable medium storing computer executable instructions operable to 
perform a method for annotating and retrieving media items, comprising: 

annotating a set of media items with a metadata to ^ciUtate metadata based retrieval of 
one or more of the set of media items; 

receiving a first query related to identifying a media item by a relationship between the 
media item and a metadata; 

retrieving a first metadata responsive to the first query; 

generating a playlist of related media items where the media items are related by one or 
more metadata items in the first metadata; 

displaying one or more media items listed in the playlist; 

receiving a second query related to identifying a media item by a relationship between the 
media item and a metadata; 

retrieving a second metadata responsive to the second query; and 
updating the playlist based on the second metadata 

52. The data structure of claim 43, the metadata further facihtates identifying the media item. 

53. The data structure of claim 43, the metadata fiirther facilitates locating the media item. 
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IX. Evidence Appendix (37 C.F.IL §41 J7(c)(l)(ix)) 
None, 

X. Related Proceedings Appendix (37 C.F.R. §41 .37(c)(l)(x)) 
None. 
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